Display apparatus, scanner, and non-transitory computer readable medium

ABSTRACT

A display apparatus includes a diffuse reflection image acquiring unit, a specular reflection image acquiring unit, a difference image acquiring unit, a reflectance distribution function calculating unit, and a display. The diffuse reflection image acquiring unit acquires a diffuse reflection image of an object surface. The specular reflection image acquiring unit acquires a specular reflection image of the object surface. The difference image acquiring unit acquires a difference image between the diffuse reflection image and the specular reflection image. The reflectance distribution function calculating unit calculates a reflectance distribution function of the object surface by using the diffuse reflection image and the difference image. The display displays a reflection color of the object surface in accordance with a change of orientation of the object surface by using the reflectance distribution function.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is based on and claims priority under 35 USC 119 fromJapanese Patent Application No. 2017-209558 filed Oct. 30, 2017.

BACKGROUND

Technical Field

The present invention relates to display apparatuses, scanners, andnon-transitory computer readable media.

SUMMARY

According to an aspect of the invention, there is provided a displayapparatus including a diffuse reflection image acquiring unit, aspecular reflection image acquiring unit, a difference image acquiringunit, a reflectance distribution function calculating unit, and adisplay. The diffuse reflection image acquiring unit acquires a diffusereflection image of an object surface. The specular reflection imageacquiring unit acquires a specular reflection image of the objectsurface. The difference image acquiring unit acquires a difference imagebetween the diffuse reflection image and the specular reflection image.The reflectance distribution function calculating unit calculates areflectance distribution function of the object surface by using thediffuse reflection image and the difference image. The display displaysa reflection color of the object surface in accordance with a change oforientation of the object surface by using the reflectance distributionfunction.

BRIEF DESCRIPTION OF THE DRAWINGS

Exemplary embodiments of the present invention will be described indetail based on the following figures, wherein:

FIG. 1 illustrates the configuration of a texture scanner according toan exemplary embodiment;

FIGS. 2A to 2C illustrate a diffuse reflection image, a specularreflection image, and a difference image therebetween acquired by thetexture scanner;

FIGS. 3A to 3C illustrate a diffuse reflection image, a specularreflection image, and a difference image therebetween acquired by acamera;

FIG. 4 illustrates the configuration of a display apparatus according toan exemplary embodiment;

FIG. 5 illustrates geometric conditions for a bidirectional reflectancedistribution function (BRDF);

FIG. 6 is a graph illustrating an example of a measured BRDF;

FIG. 7 illustrates the Phong reflection model;

FIG. 8 is a graph illustrating an example where the measured BRDF isapplied to the Phong reflection model;

FIGS. 9A and 9B illustrate images (computer graphics (CG) images)displayed on a display unit according to an exemplary embodiment;

FIGS. 10A and 10B illustrate other images (CG images) displayed on thedisplay unit according to the exemplary embodiment;

FIGS. 11A to 11D illustrate a diffuse reflection image, a specularreflection image, and two difference images according to an exemplaryembodiment;

FIGS. 12A to 12C illustrate images (CG images) displayed on the displayunit according to the exemplary embodiment;

FIG. 13 illustrates the configuration of a display apparatus accordingto another exemplary embodiment;

FIG. 14 illustrates the Torrance-Sparrow reflection model;

FIG. 15 illustrates the configuration of a texture scanner according toa modification; and

FIG. 16 illustrates a system configuration according to a modification.

DETAILED DESCRIPTION

Exemplary embodiments of the present invention will be described indetail below with reference to the drawings.

First Exemplary Embodiment

1. Configuration of Scanner (Texture Scanner)

FIG. 1 illustrates the configuration of a scanner (texture scanner) 5according to a first exemplary embodiment for acquiring an image of asurface of a planar object 10.

The texture scanner 5 optically reads the surface characteristics of theplanar object 10 and generates an image signal expressing the readresult. The image signal generated by the texture scanner 5 includes animage signal based on diffuse reflection light and an image signal basedon specular reflection light. The texture scanner 5 includes a platenglass member 12, a carriage 14, light sources 16, 18, and 20, and asensor 22.

In the texture scanner 5, each of the components shown in FIG. 1 has awidth set in the direction orthogonal to the plane of the drawing. Thisdirection is a first scanning direction of the texture scanner 5. Adirection indicated by an arrow in FIG. 1 is a second scanning directionof the texture scanner 5.

The platen glass member 12 is a transparent glass plate that supportsthe planar object 10 to be read. The platen glass member 12 mayalternatively be, for example, an acrylic plate instead of a glassplate. Although not shown, a platen cover that covers the platen glassmember 12 to block off external light may be provided such that theplanar object 10 is interposed between the platen cover and the platenglass member 12.

When the planar object 10 is to be read, the carriage 14 moves at apredetermined speed in the second scanning direction. The carriage 14contains the light sources 16, 18, and 20 therein. The light source 16is a front light source that radiates light at an incident angle of 45°,which is a first incident angle, relative to the normal direction of theplanar object 10 so as to radiate light for reading diffuse reflectionlight from the planar object 10. The light source 18 is a rear lightsource that radiates light at an incident angle of 45° relative to thenormal direction of the planar object 10 so as to radiate light forreading diffuse reflection light from the planar object 10. The lightsource 20 is a rear light source that radiates light at an incidentangle of 10°, which is a second incident angle, relative to the normaldirection of the planar object 10 so as to radiate light for readingspecular reflection light from the planar object 10.

The light source 20 is provided at a position where it does not block aprincipal ray of the reflection light. Although the incident angle ofthe light radiated by the light source 20 is 10° in this exemplaryembodiment, the incident angle may be about 5° to 10°. With regard tothe reflection light of the light radiated by the light source 20, a rayof the light traveling in the normal direction of the planar object 10is read.

It is desirable that the light source 20 radiate light at a small angle.If the angle of the light radiated by the light source 20 is relativelylarge, for example, a cover that limits the angle of the light radiatedby the light source 20 may be provided. Moreover, since the light source20 is provided for reading luster information of the planar object 10,it is desirable that the luminance be uniform and continuous as much aspossible in the first scanning direction, as compared with the lightsources 16 and 18.

Examples that may satisfy the conditions of the light source 20 includea fluorescent lamp and a rare gas fluorescent lamp (such as a xenonfluorescent lamp). The light source 20 may have multiple white lightemitting diodes (LEDs) arranged in the first scanning direction, and theluminance distribution in the first scanning direction may be madeuniform by using, for example, a diffuser.

The carriage 14 also contains therein an imaging optical system and thesensor 22. The imaging optical system is constituted of a reflectingmirror and an imaging lens and causes the sensor 22 to form images ofdiffuse reflection light and specular reflection light components fromthe planar object 10. The sensor 22 receives the diffuse reflectionlight and specular reflection light components imaged by the imagingoptical system so as to generate an image signal according to thereceived light. The sensor 22 is constituted of a light receivingelement, such as a charge-coupled-device (CCD) linear image sensor or acomplementary metal-oxide semiconductor (CMOS) image sensor, andconverts the received light into a signal expressing the magnitude ofthe received light. Moreover, the sensor 22 includes a color filter andgenerates an image signal expressing the color of the planar object 10.The sensor 22 outputs, to an external apparatus, a diffuse reflectionimage signal obtained by receiving the diffuse reflection light and aspecular reflection image signal obtained by receiving the specularreflection signal.

In a normal scanner (or an image reader), the light source 16 (or thelight source 18) radiates light at an incident angle of 45° relative tothe normal direction of the planar object 10 so as to read a diffusereflection light from the planar object 10. In contrast, the texturescanner 5 according to this exemplary embodiment additionally causes thelight source 20 to radiate light at an incident angle of 10° relative tothe normal direction of the planar object 10 so as to read specularreflection light from the planar object 10.

FIGS. 2A and 2B illustrate examples of a diffuse reflection image and aspecular reflection image of the planar object 10 obtained by thetexture scanner 5 shown in FIG. 1. The planar object 10 is a printedmaterial output based on electrophotography by varying the coverage,which is a toner amount during printing, using a cyan+silver toner, amagenta+silver toner, a magenta toner, and a cyan toner. FIG. 2Aillustrates a diffuse reflection image obtainable by a normal scanner.FIG. 2B is a specular reflection image in which a metallic lustersection of the planar object 10 is particularly glossy. In thisexemplary embodiment, a metallic luster region of the planar object 10is extracted by using these two images. Specifically, a difference imageis acquired by calculating a difference between the diffuse reflectionimage and the specular reflection image.

FIG. 2C illustrates a difference image between the diffuse reflectionimage shown in FIG. 2A and the specular reflection image shown in FIG.2B. In the image, the metallic luster region (i.e., the region includingthe silver toner in actuality) is clearly shown, and the difference inreflectance of the metallic luster is also shown.

In the texture scanner 5 shown in FIG. 1, a value obtained by dividingan incident angle by an acceptance angle is fixed at each pixel of atwo-dimensional plane so that, by calculating a difference between animage acquired under a diffuse reflection condition (i.e., incidentangle of 45°) and an image acquired under a specular reflectioncondition (i.e., incident angle of 10°), metallic luster information isproperly extracted. Specifically, by performing a simple differencecalculation, the metallic luster region and the reflectance (i.e.,specular reflectance of the two-dimensional plane) may be acquired atone time. With regard to the diffuse reflection condition (i.e.,incident angle of 45°) and the specular reflection condition (i.e.,incident angle of 10°), calibration is performed using the same whitecalibration plate. Therefore, luster information may be extracted byperforming a simple difference calculation.

FIGS. 3A to 3C illustrate images of the same planar object 10 acquiredusing a camera for a comparison and show an image acquired under thediffuse reflection condition (i.e., incident angle of 45°) and an imageacquired under the specular reflection condition (i.e., incident angleof 10°) by using a white LED. In the case of a camera, the incidentangle/acceptance angle condition varies at each pixel of thetwo-dimensional plane so that the image acquired under the specularreflection condition has uneven luster. Thus, even if a differencebetween the image acquired under the diffuse reflection condition andthe image acquired under the specular reflection condition is acquired,it is difficult to properly extract metallic luster information (i.e.,silver toner region and reflectance) of the two-dimensional plane.Therefore, in order to properly extract metallic luster information byusing a camera, it is necessary to acquire images from a larger numberof angles and estimate the metallic luster information from the acquiredimage data, which implies that a large amount of calculation isnecessary. Thus, in order to properly acquire luster information of thesurface of an object having a large surface area (e.g., a printedmaterial of an A3 sheet size), a large amount of calculation isnecessary.

2. Configuration of Display Apparatus

FIG. 4 illustrates the overall configuration according to this exemplaryembodiment. A display apparatus 25 is configured to display the textureof the planar object 10 by acquiring and processing the diffusereflection image and the specular reflection image obtained by thetexture scanner 5 shown in FIG. 1.

The display apparatus 25 includes a diffuse reflection image acquiringunit 30, a specular reflection image acquiring unit 32, a differenceimage acquiring unit 34, a diffuse reflectance distribution functioncalculating unit 36, a specular reflection distribution functioncalculating unit 38, a parameter adjusting unit 40, a reflectancedistribution function calculating unit 42, a light source informationacquiring unit 44, a camera information acquiring unit 46, a renderingunit 48, and a display unit 50.

The diffuse reflection image acquiring unit 30 and the specularreflection image acquiring unit 32 respectively acquires the diffusereflection image and the specular reflection image obtained by thetexture scanner 5. The diffuse reflection image acquiring unit 30 andthe specular reflection image acquiring unit 32 may both be connected tothe texture scanner 5 and acquire these images from the texture scanner5, or may acquire these images from a server connected to the texturescanner 5 via a network.

The difference image acquiring unit 34 acquires a difference image bycalculating a difference between the diffuse reflection image and thespecular reflection image. There are two kinds of difference images,namely, a difference image obtained by subtracting the diffusereflection image from the specular reflection image (specular reflectionimage-diffuse reflection image) and a difference image obtained bysubtracting the specular reflection image from the diffuse reflectionimage (diffuse reflection image-specular reflection image), and thedifference image acquiring unit 34 calculates at least one of thesedifference images.

The diffuse reflectance distribution function calculating unit 36calculates a diffuse reflectance distribution function of the planarobject 10 by using the diffuse reflection image. For example, inaccordance with the Lambert reflection model, the diffuse reflectancedistribution function calculating unit 36 uses a diffuse reflectancedistribution function ρ_(d)·cos θ_(i), where ρ_(d) denotes diffusereflectance with respect to incident light and θ_(i) denotes theincident angle, so as to calculate the diffuse reflectance ρ_(d) as aparameter from the diffuse reflection image.

The specular reflection distribution function calculating unit 38calculates a specular reflectance distribution function of the planarobject 10 by using the difference image. For example, in accordance withthe Phong reflection model, the specular reflection distributionfunction calculating unit 38 uses a specular reflectance distributionfunction ρ_(s)·cos^(n) γ, where ρ_(s) denotes specular reflectance, γdenotes an angle formed between a specular reflection direction and avisual line direction, and n denotes a specular reflection index, so asto calculate the specular reflectance ρ_(s) and the specular reflectionindex n as parameters from the difference image. In a case where twodifference images are acquired by the difference image acquiring unit 34and the specular reflectance distribution function is to be calculatedby using these two difference images, the specular reflectiondistribution function calculating unit 38 uses a specular reflectancedistribution function ρ_(s1)·cos^(n1) γ for the difference image(specular reflection image-diffuse reflection image) and a specularreflectance distribution function ρ_(s2)·cos^(n2) γ for the differenceimage (diffuse reflection image-specular reflection image) so as tocalculate ρ_(s1), ρ_(s2), n1, and n2 as parameters from these differenceimages.

The reflectance distribution function calculating unit 42 uses thediffuse reflectance distribution function calculated by the diffusereflectance distribution function calculating unit 36 and the specularreflectance distribution function calculated by the specular reflectiondistribution function calculating unit 38 so as to calculate areflectance distribution function for each pixel of the planar object10. For example, in accordance with the Lambert reflection model and thePhong reflection model, the reflectance distribution functioncalculating unit 42 calculates the reflectance distribution function asfollows:Reflectance Distribution Function=Diffuse Reflectance DistributionFunction+Specular Reflectance Distribution Function

Based on the reflectance distribution function calculated by thereflectance distribution function calculating unit 42, variousparameters set by the parameter adjusting unit 40, light sourceinformation (i.e., light source direction) acquired by the light sourceinformation acquiring unit 44, and camera information (i.e., visual linedirection) acquired by the camera information acquiring unit 46, therendering unit 48 renders a three-dimensional model on a virtual screenset within a virtual three-dimensional space so as tocomputer-graphically reproduce the texture of the planar object 10, andcauses the display unit 50 to display the three-dimensional model. Therendering process is widely known. For example, the rendering processmay be performed by using the radiosity technique or the ray tracingtechnique in view of interreflection.

The display apparatus 25 shown in FIG. 4 is specifically realized by acomputer equipped with one or more processors, a memory, an input-outputinterface, a communication interface, and a display unit. The processorreads a processing program stored in a nonvolatile memory, such as aread-only memory (ROM), a hard disk drive (HDD), or a solid state drive(SSD), and executes the processing program so as to realize the functionof each component. For example, the memory functions as the diffusereflection image acquiring unit 30 and the specular reflection imageacquiring unit 32 so as to store a diffuse reflection image and aspecular reflection image, and the processor executes the processingprogram so as to function as the difference image acquiring unit 34, thediffuse reflectance distribution function calculating unit 36, thespecular reflection distribution function calculating unit 38, thereflectance distribution function calculating unit 42, and the renderingunit 48. Specifically, the processor reads the diffuse reflection imageand the specular reflection image stored in the memory to calculate adifference therebetween and stores a generated difference image in thememory. Then, the processor reads the diffuse reflection image stored inthe memory to calculate a diffuse reflectance distribution function,stores the parameters thereof in the memory, reads the difference imagestored in the memory to calculate a specular reflectance distributionfunction, and stores the parameters thereof in the memory. The processorcalculates, for example, a reflectance distribution function as a sum ofthe diffuse reflectance distribution function and the specularreflectance distribution function. The steps executed in the processorare as follows.

Step a: Acquire a diffuse reflection image of the planar object 10 andstore the image in the memory.

Step b: Acquire a specular reflection image of the planar object 10 andstore the image in the memory.

Step c: Acquire a difference image between the diffuse reflection imageand the specular reflection image and store the image in the memory. Inthis case, the difference image is acquired from the texture scanner 5if the difference image is generated in and output from the texturescanner 5. If the difference image is not output from the texturescanner 5, a difference image is generated and acquired by calculating adifference between the diffuse reflection image and the specularreflection image.

Step d: Calculate a reflectance distribution function of the planarobject 10 by using the diffuse reflection image and the differenceimage.

Step e: Cause the display unit 50 to display a change in reflectioncolor of the planar object 10 caused by a difference in incident angleof light or viewing angle using the reflectance distribution function.

Step d further includes the following two steps.

Step d1: Calculate a diffuse reflectance distribution function by usingthe diffuse reflection image.

Step d2: Calculate a specular reflectance distribution function by usingthe difference image.

The one or more processors may each be constituted of a centralprocessing unit (CPU) or a graphics processing unit (GPU). The displayunit 50 is constituted of a liquid crystal display or an organicelectroluminescence (EL) display. The parameter adjusting unit 40, thelight source information acquiring unit 44, and the camera informationacquiring unit 46 may each be constituted of an input device, such as akeyboard, a mouse, and/or a touchscreen.

A computer may be connected directly to the texture scanner 5 shown inFIG. 1, or may be connected to the texture scanner 5 and the server viaa network. The computer includes a personal computer (PC), a tabletterminal, and a smartphone, and includes, for example, a tablet terminalequipped with a tilt sensor.

3. Reflectance Distribution Function

Next, the reflectance distribution function will be described.

FIG. 5 illustrates geometric conditions for a bidirectional reflectancedistribution function (BRDF). A BRDF is a function expressing the ratioof a radiance dL₀ of light reflected at a micro solid angle of a visualline direction V to an irradiance dE of light entering from a microsolid angle of a light source direction L with respect to a certainpoint x on a reflection surface. Normally, a BRDF is measured using ameasuring device capable of changing the incident angle or theacceptance angle, such as a goniophotometer. The measured value isreferred to as “measured BRDF”.

A BRDF is expressed with the following expression.

${f\left( {x,\theta_{i},\phi_{i},\theta_{o},\phi_{o}} \right)} = \frac{d\;{L_{o}\left( {\theta_{o},\phi_{o}} \right)}}{{dE}\left( {\theta_{i},\phi_{i}} \right)}$

FIG. 6 illustrates an example of a measured BRDF of a printed materialoutput based on electrophotography by varying the coverage using asilver toner. A light-source-direction vector L and avisual-line-direction vector V are on the same plane (no ϕ component),an incident angle θ_(i) is 45°, and the horizontal axis denotes anacceptance angle θ₀. A specular reflection angle is 45° when theincident angle is 45°.

According to FIG. 6, the luminance of specular reflection lightincreases with increasing coverage (i.e., metallic luster) of the silvertoner.

Next, reflection models are generated by using the measured BRDF. Thereflection models that may be used include the Lambert reflection modeland the Phong reflection model.

FIG. 7 illustrates geometric conditions for the Lambert reflection modeland the Phong reflection model.

Assuming that I_(i) denotes the intensity of incident light, ρ_(d)denotes diffuse reflectance, θ_(i) denotes an incident angle, ρ_(s)denotes specular reflectance, n denotes a specular reflection index, andγ denotes an angle formed between the specular reflection direction andthe visual line direction, the intensity I of reflection light isexpressed as follows:I=I _(i)·(ρ_(d)·cos θ_(i))+I _(i)·(ρ_(s)·cos^(n) γ)

In this reflection model, parameters including the diffuse reflectanceρ_(d), the specular reflectance ρ_(s), and the specular reflection indexn are estimated from the measured BRDF by performing a nonlinearregression analysis.

FIG. 8 illustrates a predicted BRDF to which the measured BRDF with thelow coverage shown in FIG. 6 is fitted in the example of the parameterestimation using the reflection model.

A BRDF acquiring method involves measuring the reflectance distributionby changing the incident angle and the acceptance angle, thus takingtime for acquiring data. Moreover, it is necessary to optimize thecoefficient of the reflection model by performing a nonlinear regressionanalysis on the acquired BRDF. Furthermore, if the sensor of themeasuring device is not an area sensor (e.g., in a case of aphotomultiplier), only an average BRDF within a measurement region isacquirable, thus making it difficult to acquire the BRDF of an objectsurface having an uneven BRDF for each region. Moreover, the difficultyin acquiring the BRDF of an object surface increases as the surface areaof the object surface increases.

In contrast, in this exemplary embodiment, the diffuse reflectancedistribution function is calculated from the diffuse reflection image,and the specular reflectance distribution function is calculated fromthe difference image. Since the difference image is properly extractedwith respect to only the luster section of the planar object 10, aspecular reflectance distribution function for each pixel of atwo-dimensional plane may be calculated with high accuracy based on thedifference image. Consequently, the texture of the object surface may bedisplayed by performing a simple calculation using a small amount ofimage data without having to acquire a BRDF.

Specifically, the reflectance distribution function calculating unit 42uses the diffuse reflectance distribution function obtained by thediffuse reflectance distribution function calculating unit 36 and thespecular reflectance distribution function obtained by the specularreflection distribution function calculating unit 38 so as to calculatereflection light intensity I(x, y) in a two-dimensional plane by usingthe following expression in accordance with the Lambert reflection modeland the Phong reflection model:I(x,y)=I _(i) ·{w _(d) ·G _(d)(x,y)·cos θ_(i) }+I _(i) ·{w _(s) ·{G_(s)(x,y)−G _(d)(x,y)}·cos^(n) γ}where {w_(d)·G_(d)(x, y)·cos θ_(i)} denotes a diffuse reflectancedistribution function, w_(d) denotes a diffuse reflection weightingfactor, G_(d)(x, y) denotes a diffuse reflection image, {w_(s)·{G_(s)(x,y)−G_(d)(x, y)}·cos^(n) γ} denotes a specular reflection distributionfunction, w_(s) denotes a specular reflection weighting factor, G_(s)(x,y) denotes a specular reflection image, and n denotes a specularreflection index. The difference image is a difference image (specularreflection image-diffuse reflection image), and a pixel with a negativedifference value is set as 0. The reflection light intensity I(x, y) iscalculated individually for each of R, G, and B components of thediffuse reflection image and the specular reflection image. As a result,a reflection color of the planar object 10 is calculated.

The diffuse reflectance distribution function calculating unit 36calculates diffuse reflectance from the diffuse reflection image and thediffuse reflection weighting factor, and the specular reflectiondistribution function calculating unit 38 calculates specularreflectance from the difference image and the specular reflectionweighting factor. In this reflection model, the parameters including thediffuse reflection weighting factor w_(d), the specular reflectionweighting factor w_(s), and the specular reflection index n are set inadvance to fixed values by the parameter adjusting unit 40 in accordancewith the reflection characteristics of the target object and the outputcharacteristics of the display apparatus 25. If the target object is ofthe same material, as in the case of the silver toner in FIG. 6, achange of luster similar to that of an actual object may be reproducedeven by setting the specular reflection index n to a fixed value.

In this embodiment, since the incident angle/acceptance angle conditionis fixed by using the texture scanner 5 shown in FIG. 1, a large numberof images is not necessary, unlike a case where a camera is used.Moreover, the reflectance distribution function is calculated for eachpixel of a two-dimensional plane from the diffuse reflection image andthe difference image, without having to estimate parameters byperforming a nonlinear regression analysis on an acquired BRDF.

FIGS. 9A and 9B illustrate examples of images (computer-graphics (CG)images) of the planar object 10 displayed on the display unit 50. Theplanar object 10 is a printed material output based onelectrophotography by varying the coverage using the cyan+silver toner,the magenta+silver toner, the magenta toner, and the cyan toner shown inFIG. 2. The images obtained are rendered by the rendering unit 48 basedon the light source information (light source direction) acquired by thelight source information acquiring unit 44 and the camera information(visual line direction) acquired by the camera information acquiringunit 46. A light source is disposed in the normal direction of theplanar object 10 (at an incident angle of 0°). FIG. 9A illustrates adiffuse reflection image obtained when a camera is disposed at an angleof 45° relative to the normal direction of the planar object 10 and theplanar object 10 is viewed at an angle. FIG. 9B illustrates a specularreflection image obtained when the camera is disposed in the normaldirection of the planar object 10 and the planar object 10 is viewedfrom the front. A change of reflection color, particularly, a change ofluster thereof, caused by a difference in viewing angle due to changingof the position of the camera is displayed. Specifically, a differencein a metallic luster region including silver toner and the reflectancethereof are clearly shown.

FIGS. 10A and 10B illustrate examples of images (CG images) of theplanar object 10 displayed on the display unit 50. In these examples, atablet terminal equipped with a tilt sensor is used as the displayapparatus 25 shown in FIG. 4 and displays a change of luster of theplanar object 10 caused as a result of changing the position of thelight source in accordance with the tilt of the tablet terminal. Thecamera is disposed in the normal direction of the planar object 10. FIG.10A illustrates a diffuse reflection image obtained when the tabletterminal is tilted such that light is received by the planar object 10at an angle (at an incident angle of 45°). FIG. 10B illustrates aspecular reflection image obtained when the tablet terminal is changedin angle such that light is received by the planar object 10 from thefront (at an incident angle of 0°). By changing the position of thelight source so that a change of luster of the planar object 10 causedby a difference in incident angle of light is dynamically displayed, thetexture of the planar object 10 is reproduced realistically inthree-dimensional computer graphics.

This exemplary embodiment is summarized as follows. Specifically, inorder to display the reflection color of the planar object 10, thefollowing two methods may be used.

The first method involves determining parameters of, for example, theLambert reflection model and the Phong reflection model and performingdisplay using the reflection models.

The second method involves acquiring diffuse reflection images andspecular reflection images of multiple angles and performing displaybased on image interpolation.

Of these two methods, images under an enormous number of angleconditions are necessary in the second method, resulting in an increasein the amount of calculation. On the other hand, the first method mayinclude a case where the diffuse reflectance and the specularreflectance are determined from the diffuse reflection image and thespecular reflection image acquired by the texture scanner 5, as in thisexemplary embodiment, and a case where the diffuse reflectance and thespecular reflectance are determined from the diffuse reflection imageand the specular reflection image acquired by the camera. In the case ofthe texture scanner 5, the specular reflectance for each pixel of atwo-dimensional plane is calculated from a simple difference between thediffuse reflection image and the specular reflection image. However, inthe case of the camera, such a simple difference may lead to unevenluster, thus making it difficult to properly extract luster informationof the two-dimensional plane. Therefore, it is necessary to determineand estimate the specular reflectance from a large number of images byacquiring a large number of images with the camera. As a result, theamount of calculation is smaller with the texture scanner 5 than withthe camera.

Second Exemplary Embodiment

In a second exemplary embodiment, the two difference images (specularreflection image-diffuse reflection image) and (diffuse reflectionimage-specular reflection image) are used as difference images betweenthe diffuse reflection image and the specular reflection image.

Specifically, the reflectance distribution function calculating unit 42uses the diffuse reflectance distribution function obtained by thediffuse reflectance distribution function calculating unit 36 and thespecular reflectance distribution function obtained by the specularreflection distribution function calculating unit 38 so as to calculatereflection light intensity I(x, y) in a two-dimensional plane by usingthe following expression in accordance with the Lambert reflection modeland the Phong reflection model:I(x,y)=I _(i) ·{w _(d) ·G _(d)(x,y)·cos θ_(i)}+I _(i) ·{w _(s1) ·{G _(s)(x,y)−G _(d)(x,y)}·cos^(n1) γ}−I _(i) ·{w_(s2) ·{G _(d)(x,y)−G _(s)(x,y)}·cos^(n2) γ}where {w_(d)·G_(d)(x, y)·cos θ_(i)} denotes a diffuse reflectancedistribution function, w_(d) denotes a diffuse reflection weightingfactor, G_(d) denotes a diffuse reflection image, {w_(s1)·{G_(s)(x,y)−G_(d)(x, y)}·cos^(n1) γ} denotes a first specular reflectiondistribution function, G_(s)(x, y) denotes a specular reflection image,{w_(s2)·{G_(d)(x, y)−G_(s)(x, y)}·cos^(n2) γ} denotes a second specularreflection distribution function, w_(s1) and w_(s2) denote specularreflection weighting factors, and n₁ and n₂ denote specular reflectionindices. The difference images are difference images (specularreflection image-diffuse reflection image) and (diffuse reflectionimage-specular reflection image), and a pixel with a negative differencevalue is set as 0.

The diffuse reflectance distribution function calculating unit 36calculates diffuse reflectance from the diffuse reflection image and thediffuse reflection weighting factor, and the specular reflectiondistribution function calculating unit 38 calculates specularreflectance from the two difference images and the two specularreflection weighting factors.

In this reflection model, the parameters including the diffusereflection weighting factor w_(d), the specular reflection weightingfactors w_(s1) and w_(s2), and the specular reflection indices n₁ and n₂are set in advance to fixed values by the parameter adjusting unit 40 inaccordance with the reflection characteristics of the target object andthe output characteristics of the display apparatus 25.

FIGS. 11A to 11D illustrate examples of a diffuse reflection image, aspecular reflection image, and difference images of the planar object 10obtained by the texture scanner 5 shown in FIG. 1 in accordance withthis exemplary embodiment. The planar object 10 is composed of apearl-pigment-containing material, and in these images, reflection lightfrom each pixel spreads not only in the specular reflection directionbut also in various angular directions. FIG. 11A illustrates a diffusereflection image, FIG. 11B illustrates a specular reflection image, andFIGS. 11C and 11D illustrate difference images. Specifically, FIG. 11Cillustrates a difference image (specular reflection image-diffusereflection image), and FIG. 11D illustrates a difference image (diffusereflection image-specular reflection image). Unlike the first exemplaryembodiment, since the difference image (specular reflectionimage-diffuse reflection image) has an increased number of pixels havingnegative difference values, the specular reflectance distributionfunction is calculated by adding the difference image (diffusereflection image-specular reflection image) in which the differencevalue of the difference image (specular reflection image-diffusereflection image) is a negative value. The diffuse reflectancedistribution function is calculated from the image in FIG. 11A, and thespecular reflectance distribution function is calculated from the imagesin FIGS. 11C and 11D.

FIGS. 12A to 12C illustrate examples of images (CG images) of the planarobject 10 in FIGS. 11A to 11D. A light source is disposed in the normaldirection of the planar object 10 (at an incident angle of 0°). FIG. 12Aillustrates a diffuse reflection image obtained when a camera isdisposed at an angle of 45° relative to the normal direction of theplanar object 10 and the planar object 10 is viewed at an angle. FIGS.12B and 12C illustrate specular reflection images obtained when thecamera is disposed in the normal direction of the planar object 10 andthe planar object 10 is viewed from the front. Specifically, the imageshown in FIG. 12B corresponds to the specular reflectance distributionfunction calculated based on the difference image (specular reflectionimage-diffuse reflection image) in accordance with the first exemplaryembodiment for a comparison, and the image shown in FIG. 12C correspondsto the specular reflectance distribution function calculated based onthe two difference images (specular reflection image-diffuse reflectionimage) and (diffuse reflection image-specular reflection image) inaccordance with the second exemplary embodiment.

In the image according to the first exemplary embodiment shown in FIG.12B, pixels that are brighter than those in the actual specularreflection image increase in number, whereas, in the image according tothe second exemplary embodiment shown in FIG. 12C, the luster reproducedis similar to that of an actual object since pixels with a negativedifference value of the difference image (specular reflectionimage-diffuse reflection image) are taken into consideration.

Accordingly, with respect to a material in which the specularreflectance distribution varies from micro region to micro region, thetexture of the object surface may still be displayed in accordance witha simple calculation using a small amount of image data.

Third Exemplary Embodiment

In the first and second exemplary embodiments, the texture scanner 5 isused to acquire a diffuse reflection image and a specular reflectionimage of a planar object so as to computer-graphically reproduce thetexture of the planar object. In a third exemplary embodiment, thetexture scanner 5 may be used to acquire a diffuse reflection image anda specular reflection image of a planar object so as tocomputer-graphically reproduce the texture of a three-dimensionalobject.

Specifically, as shown in FIG. 13, a shape information acquiring unit 45is added so as to acquire a three-dimensional shape model thatreproduces the texture. Based on light source information (light sourcedirection) acquired by the light source information acquiring unit 44,camera information (visual line direction) acquired by the camerainformation acquiring unit 46, and shape information acquired by theshape information acquiring unit 45, a three-dimensional model isrendered on a virtual screen set within a virtual three-dimensionalspace so that the texture of a three-dimensional object iscomputer-graphically reproduced, and the three-dimensional model is thendisplayed on the display unit 50. Consequently, a change of reflectioncolor caused when the texture of a planar object is transferred to afreely-chosen three-dimensional shape model may be displayed.

Although exemplary embodiments of the present invention have beendescribed above, the exemplary embodiments of the present invention arenot limited to these exemplary embodiments, and various modificationsare permissible.

For example, although the specular reflection index n used has the samevalue among the pixels in the two-dimensional plane, the specularreflection index n may be varied from pixel to pixel in accordance withthe following expression:I(x,y)=I _(i) ·{w _(d) ·G _(d)(x,y)·cos θ_(i) }+I _(i) ·{w _(s) ·{G_(s)(x,y)−G _(d)(x,y)}·cos^(n(x,y)) γ}

For example, the specular reflection index n may be set as a function ofthe difference image between the diffuse reflection image and thespecular reflection image in accordance with the following expression,such that n increases with increasing luminance of the difference image:n(x,y)=f(G _(s)(x,y)−G _(d)(x,y))

Furthermore, although the Phong reflection model is used in the aboveexemplary embodiments, the Torrance-Sparrow reflection model or theCook-Torrance reflection model may be used as an alternative.

FIG. 14 illustrates the Torrance-Sparrow reflection model. Assuming thatI_(i) denotes incident light intensity, w_(s) denotes a specularreflection weighting factor, G_(s)(x, y) denotes a specular reflectionimage, G_(d)(x, y) denotes a diffuse reflection image, α denotes anangle at which a bisector of an incident angle and a reflection angleforms a normal line, σ denotes surface roughness, and θ_(r) denotes areflection angle, specular reflection light intensity I_(s) (x, y) in atwo-dimensional plane is expressed as follows:

${I_{s}\left( {x,y} \right)} = {I_{i} \cdot w_{s} \cdot \left\{ {{G_{s}\left( {x,y} \right)} - {G_{d}\left( {x,y} \right)}} \right\} \cdot \frac{\exp\left( {{{- \alpha^{2}}/2}\sigma^{2}} \right)}{\cos\;\theta_{r}}}$

In this reflection model, the parameters including the specularreflection weighting factor w_(s) and the surface roughness σ are set inadvance to fixed values by the parameter adjusting unit 40 in accordancewith the reflection characteristics of the target object and the outputcharacteristics of the display apparatus 25.

With regard to the Cook-Torrance reflection model, assuming that I_(i)denotes incident light intensity, w_(s) denotes a specular reflectionweighting factor, G_(s)(x, y) denotes a specular reflection image,G_(d)(x, y) denotes a diffuse reflection image, α denotes an angle atwhich a bisector of an incident angle and a reflection angle forms anormal line, m denotes surface roughness, and θ_(r) denotes a reflectionangle, specular reflection light intensity I_(s) (x, y) in atwo-dimensional plane is expressed as follows:

${I_{s}\left( {x,y} \right)} = {I_{i} \cdot w_{s} \cdot \left\{ {{G_{s}\left( {x,y} \right)} - {G_{d}\left( {x,y} \right)}} \right\} \cdot \frac{\exp\left( {{- \tan^{2}}{\alpha/m^{2}}} \right)}{m^{2}\cos^{4}{\alpha \cdot \cos}\;\theta_{r}}}$

In this reflection model, the parameters including the specularreflection weighting factor w_(s) and the surface roughness m are set inadvance to fixed values by the parameter adjusting unit 40 in accordancewith the reflection characteristics of the target object and the outputcharacteristics of the display apparatus 25.

Furthermore, although the parameters including the diffuse reflectionweighting factor w_(d), the specular reflection weighting factor w_(s),and the specular reflection index n are set in advance to fixed valuesby the parameter adjusting unit 40, a parameter adjustment function maybe displayed on the display unit 50, such that the reflection color ofthe CG image may be varied by changing the parameters on the displayunit 50.

For example, the weighting factors may be varied, and the reflectancedistribution function may be evaluated by using a fixed evaluationfunction, such that a reflectance distribution function with arelatively high evaluation value may be selected. Then, CG images withvaried weighting factors may be sequentially displayed on the displayunit 50, and the user may select a weight considered to be optimal.

Although a difference image is calculated by the difference imageacquiring unit 34 of the display apparatus 25 in each of the aboveexemplary embodiments, as shown in FIG. 4, the texture scanner 5 shownin FIG. 1 may be equipped with an arithmetic unit that calculates adifference between a diffuse reflection image and a specular reflectionimage and outputs the difference to an external apparatus.

FIG. 15 illustrates another configuration of the texture scanner 5. Inaddition to the components of the texture scanner 5 shown in FIG. 1, thetexture scanner 5 includes an arithmetic unit 23. The arithmetic unit 23generates a difference image by calculating a difference between adiffuse reflection image and a specular reflection image, and outputsthe difference image to an external apparatus. Therefore, the texturescanner 5 outputs a diffuse reflection image, a specular reflectionimage, and a difference image from an output interface to the externalapparatus. The difference image is at least one of a difference image(diffuse reflection image-specular reflection image) and a differenceimage (specular reflection image-diffuse reflection image). The externalapparatus may be the display apparatus 25 or an external server, or maybe a portable memory, such as a universal-serial-bus (USB) memory or asecure-digital (SD) card.

Furthermore, a CG image created and displayed by the display apparatus25 shown in FIG. 4 may be appropriately registered in the server via anetwork, so as to form a CG-image database.

FIG. 16 illustrates a system configuration according to a modification.The texture scanner 5, a server 100, and the display apparatus 25 areconnected to one another in a data exchangeable fashion via a network. Adiffuse reflection image, a specular reflection image, and a differenceimage obtained by the texture scanner 5 are transmitted to the server100 via the network and are stored in the server 100. The displayapparatus 25 accesses the server 100 to acquire the diffuse reflectionimage, the specular reflection image, and the difference image,calculates a reflectance distribution function, and causes the displayunit 50 to display the reflection color. A CG image displayed by thedisplay apparatus 25 is transmitted to the server 100 via acommunication interface and the network and is stored in the server 100,and is also transmitted to another apparatus when requested from thedisplay apparatus. The server 100 may display a list of CG imagesuploaded from the display apparatus 25 in another display apparatus inthe form of thumbnail images, and may allow for selection of the CGimages in the other display apparatus.

The foregoing description of the exemplary embodiments of the presentinvention has been provided for the purposes of illustration anddescription. It is not intended to be exhaustive or to limit theinvention to the precise forms disclosed. Obviously, many modificationsand variations will be apparent to practitioners skilled in the art. Theembodiments were chosen and described in order to best explain theprinciples of the invention and its practical applications, therebyenabling others skilled in the art to understand the invention forvarious embodiments and with the various modifications as are suited tothe particular use contemplated. It is intended that the scope of theinvention be defined by the following claims and their equivalents.

What is claimed is:
 1. A display apparatus comprising: a processorconfigured to acquire an image signal of an object surface, wherein theimage signal comprising a diffuse reflection image of the object surfaceand a specular reflection image of the object surface, acquire adifference image between the diffuse reflection image and the specularreflection image, wherein the difference image acquired by the processorincludes an image obtained by subtracting the diffuse reflection imagefrom the specular reflection image and an image obtained by subtractingthe specular reflection image from the diffuse reflection image, andcalculate a reflectance distribution function of the object surface byusing a diffuse reflectance distribution function calculated from thediffuse reflection image, a first specular reflectance distributionfunction calculated from the image obtained by subtracting the diffusereflection image from the specular reflection image, and a secondspecular reflectance distribution function calculated from the imageobtained by subtracting the specular reflection image from the diffusereflection image; and a display that displays a reflection color of theobject surface in accordance with a change of orientation of the objectsurface by using the reflectance distribution function.
 2. The displayapparatus according to claim 1, wherein the reflectance distributionfunction is calculated from the diffuse reflectance distributionfunction, a function obtained by multiplying the first specularreflectance distribution function by a weight w1, and a functionobtained by multiplying the second specular reflectance distributionfunction by a weight w2, and wherein the reflectance distributionfunction is calculated while varying the weight w1 and the weight w2. 3.The display apparatus according to claim 1, wherein the diffusereflection image and the specular reflection image are images obtainedby scanning the object surface by using a scanner including a firstlight source and a second light source, the first light source radiatinglight onto the object surface at a first incident angle, the secondlight source radiating light onto the object surface at a secondincident angle different from the first incident angle.
 4. The displayapparatus according to claim 3, wherein the first incident angle is 45°,and the second incident angle ranges between 5° and 10°.
 5. A scannercomprising: a first light source that radiates light onto an objectsurface at a first incident angle; a second light that radiates lightonto the object surface at a second incident angle different from thefirst incident angle; a processor configured to acquire an image signalof the object surface, wherein the image signal comprising a diffusereflection image of the object surface and a specular reflection imageof the object surface, calculate a difference image between the diffusereflection image of the object surface and the specular reflection imageof the object surface, the diffuse reflection image being obtained as aresult of the object surface being irradiated with the light at thefirst incident angle, the specular reflection image being obtained as aresult of the object surface being irradiated with the light at thesecond incident angle, wherein the first incident angle is 45° to theobject surface, and the second incident angle ranges between 5° and 10°to the object surface; and an output that outputs the diffuse reflectionimage, the specular reflection image, and the difference image.
 6. Anon-transitory computer readable medium storing a program causing acomputer to execute a process, the process comprising: acquire an imagesignal of an object surface, wherein the image signal comprising adiffuse reflection image of the object surface and a specular reflectionimage of the object surface; storing the diffuse reflection image andthe specular reflection image in a memory; acquiring a difference imagebetween the diffuse reflection image and the specular reflection image,wherein the difference image includes an image obtained by subtractingthe diffuse reflection image from the specular reflection image and animage obtained by subtracting the specular reflection image from thediffuse reflection image, and storing the difference image in thememory; calculating a reflectance distribution function of the objectsurface by using a diffuse reflectance distribution function calculatedfrom the diffuse reflection image, a first specular reflectancedistribution function calculated from the image obtained by subtractingthe diffuse reflection image from the specular reflection image, and asecond specular reflectance distribution function calculated from theimage obtained by subtracting the specular reflection image from thediffuse reflection image; and causing a display to display a reflectioncolor of the object surface in accordance with a change of orientationof the object surface by using the reflectance distribution function.